Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add rejection sampling analysis #253

Draft
wants to merge 53 commits into
base: main
Choose a base branch
from
Draft

Add rejection sampling analysis #253

wants to merge 53 commits into from

Conversation

vwxyzjn
Copy link
Collaborator

@vwxyzjn vwxyzjn commented Aug 13, 2024 

🚀 Dataset loaded, starting to analyze...
chosen:    mean length = 163.50656107997827
rejected:  mean length = 150.02915225944915
reference: mean length = 811.4574606202721
key='allenai/llama-3-tulu-2-8b-uf-mean-rm'
chosen:    mean score = 26.952228895910338
           std score = 3.0159979740827905
rejected:  mean score = 23.47721361929176
           std score = 3.226889784341013
reference: mean score = 30.24208389637402
           std score = 4.438492691848654
reference is better than chosen

So I realized the issue is that we generate only 53 tokens by default. We should change it to a much more sensible number, prob 2048 @nouhadziri

open-instruct/open_instruct/generation.py

Line 43 in 1515330

response_length: int = 53

Let's merge this after your PR #252 and I can add documentation

@nouhadziri
move rejection sampling within its own directory
9035816
@nouhadziri
move rejection sampling within its own directory
e97eb4e
@nouhadziri
add api-based models to generating completions
4bfc79b
@nouhadziri
debug
7a0abc4
@nouhadziri
debug
034259d
@nouhadziri
debug
b5990ef
@nouhadziri
debug
d44e1cb
@nouhadziri
debug
1c99e32
@nouhadziri
do not apply tokenization with chat template
6013160
@nouhadziri
change the tokenizer
c47a16e
@nouhadziri
change the tokenizer
91fd8b8
@nouhadziri
fix template
7093b10
@nouhadziri
fix arg issue
805fcac
@nouhadziri
add mode to arg
f93e738
@nouhadziri
add mode to arg
77d257b
@nouhadziri
change the skill
b2a4b83
@nouhadziri
remove main from api_generate.py
b80894a
@nouhadziri
modify template chat
fca6f6d
@nouhadziri
modify template chat
6541c9f
@nouhadziri
return only text
1630b4b
@nouhadziri
return only text
ebacf9e
@nouhadziri
return only text
45d14b3
@nouhadziri
fix dict
60a7a87
@nouhadziri
fix dict
33e8f8c
@nouhadziri
fix dict
9d99744
@nouhadziri
fix dict
7ce7419
@nouhadziri
fix dict
f3f1466
@nouhadziri
Merge branch 'main' into llm-as-judge
97b812e
@nouhadziri
fix dict
7fe90fd
@nouhadziri
add LM as a scorer
fe0cfe3
@nouhadziri
Copy link
Contributor

This explains the performance!! I haven't had a chance to play with different generation parameters yet but increasing the length will definitely boost numbers.

@nouhadziri
Copy link
Contributor

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vwxyzjn @nouhadziri